ISBN: 978-972-8939-88-5 © 2013 lADIS 


THE DESIGN OE THE TEST EORMAT EOR TABLET 
COMPUTERS IN BLENDED LEARNING ENVIRONMENTS: 
A STUDY OE THE TEST APPROACH- AVOIDANCE 
TENDENCY OE UNIVERSITY STUDENTS 


Takeshi Kitazawa 

School of Motivation and Behavioral Sciences, Tokyo Future University 


ABSTRACT 

This study analyzed effective test formats that utilized tablets for tests in university information basic subjects in blended 
learning environments. Specifically, three types of test were created: (1) multiple-choice, (2) fill-in-the-blank, and (3) a 
mixture of multiple-choice and fill-in-the-blank. An analysis focusing on university students’ approach-avoidance 
tendency was conducted to see how the question formats affected the students’ motivations toward the tests. The results 
showed that regardless of the test approach-avoidance tendency, the students had a low perception of their understanding 
of the content of multiple-choice tests, even though they reported that they did not feel burdened in answering the 
questions. Additionally, although the students had a high perception of knowledge retention with fill-in-the-blank 
questions, their willingness to answer the questions and the percentage of questions they answered correctly were low. 
On the other hand, students had a high perception of continuity and a high percentage of correct answers on the test with 
a mixture of multiple-choice and fill-in-the-blank questions. Additionally, they had a high perception of knowledge 
retention in the mixed test, suggesting that this format had the strongest effect among the three test formats. 
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1. INTRODUCTION 

As a result of faculty development being made compulsory, Japanese universities have been asked to 
improve their classes with the aim of promoting student learning outside of class (i.e., after-class hours 
learning) as well as improving student academics and knowledge retention (Central Education Council, 2008). 
Furthermore, for information technology (IT), a subject that became compulsory for high school in 2003, the 
number of subjects and teaching content were revised in the 2013 academic year. Therefore, it is necessary to 
improve information education classes that are aimed at new students enrolled in IT courses. Blended 
learning, which integrates face-to-face lessons with an e-learning system, has been proposed as a method to 
resolve this issue (e.g., Akkoyunlu et al., 2008). It is anticipated that constructing an environment in which 
students can access an e-learning system using mobile terminals will encourage after-class hours learning, as 
it will allow student to engage in learning anywhere and at any time. 

Moreover, administering tests effectively can be considered as a method of knowledge retention (Bloom, 
1971). For example, when teachers give advance notice of a test, students are externally motivated to study, 
which promotes their learning (Kuramitsu, 1980). Based on this finding, one can expect that asking students 
to take tests in a mobile environment outside of class hours will produce a knowledge retention effect and 
promote after-class hours learning. However, since learners’ perception of tests, or their perception of the 
objective of conducting tests and their roles, affects their learning behaviors, it is important to administer 
tests that take into account learners’ approach-avoidance tendencies (Suzuki 201 1). 

Based on the above findings, the authors previously constructed a blended learning environment by 
integrating face-to-face classes and an e-learning system that utilized mobile terminals, focusing on the test 
approach-avoidance tendency in first-year university information education classes. Having constructed this 
environment, the authors explored the motivations toward and learning effects of tests conducted using 
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mobile terminals outside of class hours; however, an analysis of the validity of test formats has not yet been 
conducted (Kitazawa et al., 2012). Therefore, this study experimentally constructed a blended learning 
environment by integrating face-to-face classes with an e-learning system using mobile terminals in first-year 
university information education classes and gave tests using a tablet (iPad), a type of a mobile terminal. This 
study then aimed to explore the relations between test formats and the test approach-avoidance tendency of 
university students and their motivations toward tests. 


2. METHOD 

2.1 Participants 

The participants were 60 Japanese university students (30 men and 30 women). The research was conducted 
on October 14, 2012. The students attended a lecture on the introductory section of the course “Introduction 
to Information Science” was held. The lecture lasted approximately 15 minutes; it was designed to resemble 
an information education class held during the first year of the university course. 

2.2 Administering Tests via Tablets 

After the lecture was over, the students took a test using a tablet terminal. The tests were distributed during 
after-class hours. The tablet terminal used in this study was the iPad (3rd generation), 16 GB, Wi-Fi model. 



Figure 1 . A mixed test format of multiple-choice and fill-in-the-blank questions 

Although there are various types of test formats, this study focuses on multiple-choice and fill-in-the- 
blank formats, which are considered easy to use autonomously. Each participant took three tests, each with a 
different format; (1) multiple-choice, (2) fill-in-the-blank, and (3) a mixture of multiple-choice and fill-in- 
the-blank questions (Figure 1). In consideration of the order effect, the participants were divided into three 
groups, with the orders of test taking different for each group. Regarding the number of questions, since five 
questions seemed to be too few (Kitazawa et al., 2012), 10 questions were created for each question format in 
this study. Each of the three tests took approximately 20 minutes to complete. 

2.3 Questionnaire 

To analyze the university students’ test approach-avoidance tendency, the test approach-avoidance tendency 
scale (a 7-item scale totaling 10 questions) developed by Suzuki (2011) was used as a pre-questionnaire. A 
post-questionnaire survey (a 5-item scale totaling 20 questions) was administered to assess the students’ 
“burdens of taking tests,” “willingness to take tests,” and “self-efficacy” as their motivation regarding tests 
(approx. 10 minutes). The data were analyzed by an overall trend analysis comparing them with the median 
values. A t-test was also conducted to analyze the differences between approach-avoidance tendencies. 
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3. RESULTS 

3.1 Pre-questionnaire 

A principal component analysis of the results of the test approach-avoidance scale with factors limited to two 
was conducted with the data of 58 out of the 60 participants, after excluding two participants with missing 
data. The numbers of people with a test-approach tendency or a test-avoidance tendency, respectively, were 
counted. There were 29 people with a test-approach tendency and 29 with a test-avoidance tendency. 

3.2 The Percentage of Correct Answers in Tests 

A two-way factorial analysis of variance was conducted on the test-approach and test-avoidance tendencies 
for the correct answer rates of the tests for each format of multiple -choice questions (average of test 
approach: 80.7, average of test avoidance: 77.9), fill-in-the-blank questions (average of test approach; 55.9, 
average of test avoidance: 54.8), and mixture (average of test approach; 76.2, average of test avoidance: 74.5). 
The results showed a significant main effect of test formats (f(2,112) = 32.7, p < .01). When Tukey’s 
multiple comparison test was conducted thereafter, a significant difference was found between the average 
values of multiple-choice and fill-in-the-blank questions (p < .01), and between fill-in-the-blank and a mix of 
multiple-choice and fill-in-the-blank questions (p < .01) for both test approach tendency and avoidance 
tendency. Furthermore, no significant difference was found between test approach tendency and avoidance 
tendency. 

3.3 Post-questionnaire 

Table 1 shows the results of the post-test questionnaire survey on the burden of test-taking, willingness 
toward tests, and self-efficacy. To analyze the overall trends, comparison analyses were conducted with a 
median value (3.0) using a t-test for the average values of each item. A significant difference was found for 
15 items at below the .01 significance level. A significant difference was found for 2 items at the .10 
significance level. 

Among the items related to multiple-choice questions, items such as (1) “Taking tests with only multiple- 
choice questions is a burden (reversed item)” (average value: 4.22) (p < 0.01) had significantly high values. 
This shows that most students do not think taking tests with only multiple-choice questions is a burden. On 
the other hand, items such as (16) “By taking tests with only multiple -choice questions, I have become able 
to explain to others what I have learned in this class (average: 2.19) (p < .001)” had significantly low values. 
This shows students perceive that tests with only multiple-choice questions do not promote knowledge 
retention and sufficient understanding. Concerning the items related to fill-in-the-blank questions, items such 
as (8) “Tests with only fill-in-the-blank questions lead to knowledge retention (average: 3.93) (p < .001)” had 
significantly high values. This shows that students that perceive tests with a fill-in-the-blank format achieve 
knowledge settlement. On the other hand, items such as (2) “Taking tests with only fill-in-the-blank 
questions is a burden (reversed item) (average: 3.93) (p < .001)” had significantly low values. This suggests 
that the fill-in-the-blank format is a burden and has little test continuity. 

Furthermore, concerning the post-test survey questionnaire results, the t-test analysis did not show a 
significant difference in any item when the difference between test approach tendency and avoidance 
tendency was analyzed. Therefore, when tests were distributed during after-class hours, using a tablet within 
an autonomous testing situation, among the three test types, the mixture of multiple-choice and fill-in-the- 
blank questions showed test continuity and knowledge retention effects, regardless of whether the student 
tendencies were test-approach or avoidance. 


4. CONCLUSION 

This study analyzed the effectiveness of test formats that utilized tablets for tests in university information 
foundation subjects in a blended learning environment. Specifically, three types of test were created; (1) 
multiple-choice, (2) fill-in-the-blank, and (3) a mixture of multiple -choice and fill-in-the-blank. An analysis 
focused on university students’ approach-avoidance tendency was conducted to investigate how the question 
formats affected students’ motivation toward the tests. The results showed that regardless of the test 
approach-avoidance tendency, students showed a low understanding of the content of multiple-choice tests. 
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even though they did not feeling burdened in taking them. Additionally, for fill-in-the-blank tests, although 
the students had high perceptions of their knowledge retention, their willingness to answer the questions and 
the percentage of questions they answered correctly were low. On the other hand, students had a high 
perception of continuity and the percentage of correct answers was high on the test with a mixture of 
multiple-choice test and fill-in-the-blank questions. Additionally, they also perceived the mixed test to give 
them high knowledge retention, suggesting that this format had the strongest effect among the three. 

Future research should include exploring a valid number of questions. Furthermore, since smartphones 
have become prevalent as mobile terminals, it is necessary to investigate the difference between smartphones 
and the tablets used in this study (iPad: 3rd generation). In addition, a long-term study in actual classroom 
environments, and not just within the experiment environment, is required. 


Table 1. Results of the post-questionnaire (Comparison analysis with medium values) 


Items 

Avg. 

SD 


1 . Taking tests with only multiple-choice questions is a burden (reversed item). 

4.22 

.65 

*** 

Tests that combine multiple-choice and fill-in-the-blank questions lead to knowledge retention. 

3.95 

.71 

*** 

8. Tests with only fill-in-the-blank questions lead to knowledge retention. 

3.93 

.99 

*** 

14. Tests with only fill-in-the-blank questions make me feel that I have sufficiently understood what I 
have learned in this class. 

3.88 

.94 

*** 

4. Tests with only multiple-choice questions make me answer the questions willingly. 

3.83 

.96 

*** 

10. Tests with only multiple-choice questions are easy to answer. 

3.78 

1.04 

*** 

15. Tests with both multiple-choice questions and fill-in-the-blank questions make me feel that I have 
sufficiently understood what I have learned in this class. 

3.78 

.86 


17. By taking tests with only fill-in-the-blank questions, I have learned to explain to others what I have 
learned in this class. 

3.67 

1.00 

*** 

12. Tests with both multiple-choice questions and fill-in-the-blank questions are easy to answer. 

3.50 

.88 

*** 

6. Tests with both multiple-choice questions and fill-in-the-blank questions make me answer the 
questions willingly. 

3.43 

.92 

** 

18. By taking tests with both multiple-choice questions and fill-in-the-blank questions, I have become 
able to explain to others what I have learned in this class. 

3.43 

.94 

** 

3. Taking tests with both multiple-choice and fill-in-the-blank questions is a burden (reversed item). 

3.17 

.94 


19. I sufficiently understood what I learned in this class. 

3.12 

.94 


11. Tests with only fill-in-the-blank questions are easy to answer. 

3.02 

1.12 


7. Tests with only multiple-choice questions lead to knowledge retention. 

2.72 

1.07 

-1- 

13. Tests with only multiple-choice questions make me feel that I have sufficiently understood what I 
have learned in this class. 

2.71 

1.14 

-1- 

5. Tests with only fill-in-the-blank questions make me answer the questions willingly. 

2.64 

.93 

** 

20. I am confident of the content I learned in this class. 

2.57 

.84 

*** 

2. Taking tests with only fill-in-the-blank questions is a burden (reversed item). 

2.24 

.94 

*** 

16. By taking tests with only multiple-choice questions, I have become able to explain to others what I 
have learned in this class. 

2.19 

.89 

*** 


*** p < .001; ** p < .01; -I- p < .10 
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